Bismut [1] first introduced linear backward stochastic differential equations (BSDEs in short) as the adjoint equation of the classical stochastic optimal control problem. In 1990, Pardoux and Peng firstly proved the existence and uniqueness of nonlinear BSDEs with Lipschitz condition [2]. Since then, the theory of BSDEs has been studied by many researchers and applied in a wide range of areas, such as in stochastic optimal control and mathematical finance [3, 4]. When a BSDE is coupled with a (forward) stochastic differential equation (SDE in short), the system is usually called a forward-backward stochastic differential equation (FBSDE in short). We can refer to the literatures in [5, 6, 7, 8, 9, 10] which studied the existence, uniqueness and the applications of coupled or fully-coupled FBSDEs.

Planning robot contact often requires reasoning over a horizon to anticipate outcomes, making such planning problems computationally expensive. In this letter, we propose a learning framework for efficient contact planning in real-time subject to uncertain contact dynamics. We implement our approach for the example task of robot air hockey. Based on a learned stochastic model of puck dynamics, we formulate contact planning for shooting actions as a stochastic optimal control problem with a chance constraint on hitting the goal. To achieve online re-planning capabilities, we propose to train an energy-based model to generate optimal shooting plans in real time. The performance of the trained policy is validated %in experiments both in simulation and on a real-robot setup. Furthermore, our approach was tested in a competitive setting as part of the NeurIPS 2023 Robot Air Hockey Challenge.

Most existing neural network-based approaches for solving stochastic optimal control problems using the associated backward dynamic programming principle rely on the ability to simulate the underlying state variables. However, in some problems, this simulation is infeasible, leading to the discretization of state variable space and the need to train one neural network for each data point. This approach becomes computationally inefficient when dealing with large state variable spaces. In this paper, we consider a class of this type of stochastic optimal control problems and introduce an effective solution employing multitask neural networks. To train our multitask neural network, we introduce a novel scheme that dynamically balances the learning across tasks. Through numerical experiments on real-world derivatives pricing problems, we prove that our method outperforms state-of-the-art approaches.

In this paper, we carry out numerical analysis to prove convergence of a novel sample-wise back-propagation method for training a class of stochastic neural networks (SNNs). The structure of the SNN is formulated as discretization of a stochastic differential equation (SDE). A stochastic optimal control framework is introduced to model the training procedure, and a sample-wise approximation scheme for the adjoint backward SDE is applied to improve the efficiency of the stochastic optimal control solver, which is equivalent to the back-propagation for training the SNN. The convergence analysis is derived with and without convexity assumption for optimization of the SNN parameters. Especially, our analysis indicates that the number of SNN training steps should be proportional to the square of the number of layers in the convex optimization case. Numerical experiments are carried out to validate the analysis results, and the performance of the sample-wise back-propagation method for training SNNs is examined by benchmark machine learning examples.

In this paper, we mainly focus on solving high-dimensional stochastic Hamiltonian systems with boundary condition, which is essentially a Forward Backward Stochastic Differential Equation (FBSDE in short), and propose a novel method from the view of the stochastic control. In order to obtain the approximated solution of the Hamiltonian system, we first introduce a corresponding stochastic optimal control problem such that the extended Hamiltonian system of the control problem is exactly what we need to solve, then we develop two different algorithms suitable for different cases of the control problem and approximate the stochastic control via deep neural networks. From the numerical results, comparing with the Deep FBSDE method developed previously from the view of solving FBSDEs, the novel algorithms converge faster, which means that they require fewer training steps, and demonstrate more stable convergences for different Hamiltonian systems.

In this paper, we introduce an efficient computational framework to quantify the uncertainty of a class of deep neural networks (DNNs). The DNN has emerged from machine learning and becomes one of the most extensively studied research topics in scientific computing and data science. As a type of artificial neural network with multiple layers, the DNN is capable to model complex systems and its applications cover wide range among various scientific and engineering disciplines. However, despite phenomenal success, the deterministic output of DNNs can not produce probabilistic predictions for the uncertain nature of scientific research. Therefore, it's difficult to apply DNNs to solve real-world scientific problems since there are typically uncertainties involved in scientific models and observed data are always perturbed by noises.

We propose a novel reformulation of the stochastic optimal control problem as an approximate inference problem, demonstrating, that such a interpretation leads to new practical methods for the original problem. In particular we characterise a novel class of iterative solutions to the stochastic optimal control problem based on a natural relaxation of the exact dual formulation. These theoretical insights are applied to the Reinforcement Learning problem where they lead to new model free, off policy methods for discrete and continuous problems.